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@ Graphics system. 



@ A graphics system connprises an input device, 
and a computer for detecting changes in the position 
of the Input device relative to a surface, and a 
monitor (63) for displaying patterns which follow the 
movement of the Input device. The computer cal- 
culates at least one differential derivative with re- 
spect to time of the input device position and con- 
trols characteristics of the display patterns in depen- 
dence upon the or each calculated derivative. 
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GRAPHICS SYSTEM 



The present invention relates to a graphics 
system. 

Computer graphics systems are known which 
convert the movements of a pen-like input across a 
tablet Into patterns on a screen, the patterns cor- 
responding in shape to the movements of the input 
device relative to the tablet. Variations of this basic 
system are also known in which for example the 
input device is pressure sensitive such that the 
width of the pattern is a function of the applied 
pressure, or the input device is In the form of a 
"mouse'* which is movable on a ball across a flat 
surface, rotations of the ball controlling the configu- 
ration of the resultant pattern. 

Such systems must respond rapidly to move- 
ments of the Input device if the user is to be able 
to exercise adequate control. If changes to the 
displayed image tagged significantly behind the 
movements of the input device causing those 
changes the user would lose the movement^image 
change visual feedback that is essential. 

Systems are available which provide an ade- 
quate rate of response to input device movement 
but users still feel inhibited by the system char- 
acteristics which are considered not to adequately 
emulate the expressive qualities of images pro- 
duced using traditional physical media. For exam- 
ple, the type of mark produced when painting with 
a soft brush is completely different from that pro- 
duced when using pen and ink, and the "feel" of 
the different implements is very different. The abil- 
ity of an artist to express himself or herself in the 
visual arts appears to be significantly affected by 
the behaviour of the media used. Available systems 
do not adequately emulate the dynamic behaviour 
of traditional media. 

It is an object of the present invention to pro- 
vide an improved computer graphics computer 
system which obviates or mitigates the problems 
outlined above. 

According to one aspect of the present inven- 
tion there is provided a graphics system compris- 
ing an Input device, means for detecting changes 
in the position of the input device relative to a 
surface, and means for displaying patterns which 
follow the movement of the Input device, wherein 
means are provided for determining at least one 
differential derivative with respect to time of the 
input device position, and means are provided for 
controlling characteristics of the displayed patterns 
in dependence upon the or each determined de- 
rivative. 

The determined derivative may be velocity 
(first derivative), acceleration (second derivative) or 
higher order derivatives. The determined deriva- 



tives are then used to control for example the 
shape, width, density, texture and colour of the 
resultant visual effect. 

Preferably, the velocity and acceleration of the 

5 input device are detected and used to control the 
displayed pattern. Inertial effects can also be simu- 
lated by considering the simulated mass of a com- 
ponent of an input device and its acceleration. 
Erratic or "jerky" movements can also be detected 

JO by monitoring the rate of change of the accelera- 
tion to enable the simulation of the results of such 
movements. Thus many of the characteristics 
which give traditional media a unique "feel" can be 
simulated. 

75 Preferably, the input device provides three di- 
mensional information, that is X and Y information 
corresponding to position on a surface and Z in- 
formation corresponding to pressure or possibly 
vertical displacement of the Input device. Scalar 

20 and vector quantities can then be calculated for 
one or more of the three dimensions corresponding 
to velocity, acceleration etc. to enable special vi- 
sual effects to be achieved corresponding to those 
familiar to users of traditional media. For example, 

25 when considering the simulation of effects obtained 
using a brush, splatter and dripping effects can be 
simulated. Such effects can be triggered in re- 
sponse to accelerations above a predetermined 
threshold, for example by monitoring the angular 

30 change in the direction of the velocity vectors be- 
tween successive sampling intervals or by monitor- 
ing angular acceleration in three dimensions. 

The derivatives could be determined directly 
for example using an accelerometer in the case of 

36 acceleration but usually the derivatives will be cal- 
culated when the system Is Implemented in a suit- 
ably programmed computer. 

In accordance with a second aspect of the 
present invention, a graphics system comprises an 

40 input device, means- for detecting changes in the 
position of the input device relative to a surface, 
means for displaying patterns which follow the 
movement of the input device, and storage means 
for storing data defining the form of a simulated 

45 surface on which the patterns are being drawn, 
wherein the means for displaying patterns Is re- 
sponsive to movement of the input device and the 
stored data to simulate the effect of the underlying, 
simulated surface on the pattern being drawn. 

60 This aspect of the invention, which is particu- 
lariy useful In combination with the first aspect of 
the invention allows real effects to be simulated 
electronically such as the slope of a surface on 
which the pattern is t>eing drawn. Thus, the data 
stored by the storage means can define the effect 
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of gravity on the coloured medium which is being 
modelled while another effect which can be . mod- 
elled is the effect of the thickness of the previously 
laid down medium such as simulated ink on a 
subsequent pass of the input device. 

Embodiments of a computer graphics system 
according to the present invention will now be 
described, by way of example, with reference to 
the accompanying drawings, in which; 

Figure 1 is a representation of a three-di- 
mensional scalar and vector field derivable from a 
pressure sensitive input device; 

Figure 2 is a flow chart illustrating the mode 
of operation of an embodiment of the present in- 
vention; 

Figure 3 is a flow chart illustrating field mod- 
elling in the embodiment of Figure 2; 

Figure 4 illustrates data derivable from the 
embodiment of Figure 2 to serve as the basis for 
generating predetermined effects; 

Figure 5 illustrates effects which can be gen- 
erated using the embodiment of Figure 2; 

Figure 6 illustrates the generation of inertial 
effects using the embodiment of Figure 2; 

Figure 7 illustrates three-dimensional model- 
ling to give a three-dimensional field database; 

Rgure 8 is a flow diagram illustrating one 
example of the mode of operation of the system; 
and, 

Figure 9 illustrates the appearance of a 
"drip" generated by the system when operated in 
accordance with the Figure 8 flow diagram. 

Referring to Rgure 1, the illustration schemati- 
cally represents the movement of a pressure sen- 
sitive input device or pen (not shown) of conven- 
tional form over a square tablet 1 , the pen following 
the line 2. In known manner, the position in terms 
of X and y coordinates relative to the tablet of the 
pen tip is output from the tablet, and the pressure 
on the pen tip is output from the pen and can be 
considered as representing a third dimension Z. 

Thus the system provides X. Y and Z coordi- 
nates which are conventionally used to trace a line 
on a screen of a monitor (not shown) correspond- 
ing In shape to the shape of the line 2 and having a 
width related to pen tip pressure. Alternative sys- 
tems to the pen and tablet system, described pro- 
duce two and three-dimensional information in a 
similar manner to produce similar screen effects. 

In a conventional system, the frame store de- 
fining pixels on a one to one basis with the monitor 
screen is continually updated with the Information 
from the tablet and thereafter controls the monitor 
display in a conventional manner. 

With the known arrangements, the screen ef- 
fects produced are intended to be independent of 
the speed at which the input device moves and its 
acceleration. If the system response time is rela- 



tively slow some speed-related effects may be- 
come apparent, simply because the system cannot 
"keep up with" the input device movement. Such 
effects are however not intended. 

5 In accordance with one aspect of the present 

invention, at least one differential derivative with 
respect to time is calculated by the system and 
used to adjust the imago resulting from movement 
of the input device. This can be done in a number 

10 of ways as described below. 

Referring to Figure 2, a flow chart is shown 
Illustrating the operation of one embodiment of the 
invention. Input coordinate information 50 In three 
dimensions corresponding to the movement of an 

75 input device such as a simulated paint brush is 
processed in accordance with a predetermined cal- 
culation procedure 51 which is controlled by a 
timing signal from a clock 52. The timing signal 
determines the period between successive evalu- 

20 ations of X, Y and Z (sampling intervals) and the 
calculation procedure 51 derives the first, second 
and third derivatives with respect to time of X, Y 
and Z. 

The absolute position (X,Y,2) 53 in 3D space 
25 (or the absolute position (X,Y) in 2D space together 
with a pressure value (2)) is provided to a calcula- 
tion routine 54, as is the velocity 55 (the vector 
sum of the first derivatives), the acceleration 56 
(the vector sum of the second derivatives), and the 
30 rate of change of acceleration 57 (the vector sum 
of the third derivatives). The sign of the accelera- 
tion 58 and of the rate of change of acceleration 59 
are also provided. This enables the effect of the 
"brush" on the "sheet" to be modelled. This mod- 
36 elling may be achieved independently of any pre- 
vious patterns which have been applied to the 
sheet or can take account of the previous patterns 
stored in a three-dimensional field model 61 . If this 
model is used then the model is updated following 
40 each calculation 54. The use of the model will be 
explained in more detail below with reference to 
Figures 3 and 7. 

The media to be emulated, e.g. soft brush and 
paint, pen and ink etc. is modelled in a media 
45 behaviour database 60 which defines various fixed 
parameters such as brush shape. The field and 
media databases provide data to enable the effect 
of moving the brush to be determined and re- 
corded in a real time dynamic database 62 (the 
50 frame store) to generate appropriate image data for 
display on an output device 63 such as the screen 
of a monitor. 

The various elements shown In Figure 2 will 
normally be implemented on a suitably pre- 
ss grammed computer with attached stores although 
hard wired circuits could also be used. 

Figure 3 illustrates the operation of the calcula- 
tion routine 54 which enables the field model 61 to 
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incorporate the results of previous inputs into a 
basic field model. For example, a pre-defined 
three-dimensional field model 61 can be estab- 
lished and stored corresponding to a flat sheet of 
paper inclined to the vertical at a predetermined 
angle corresponding to the position adopted by an 
artists easel. Infomnation relating to the position, 
velocity etc. of the input device is submitted to the 
field transformation calculation routine 54 which 
takes into account the pre-defined three-dimen- 
sional field model 61 and the media behaviour 
database 60 to produce desired effects, e.g. paint 
slippage in the direction of the bottom edge of the 
simulated paper, and provides transformed vector 
and scalar outputs 64 accordingly. In the illustrated 
embodiment, the transformed outputs are fed back 
to the field model 61 to update the predefined 
model, thereby enabling subsequent inputs to be 
affected by earlier inputs. For example, thick hori- 
zontal layers of paint could affect the slippage of 
overlying vertical layers of paint, and this can be 
taken into account in the updated field model. It Is 
not necessary however for the field model to be 
updated by a feedback input in this way. 

Referring now to Figure 4, one example of a 
graphic effect which can be achieved in accor- 
dance with the invention is illustrated. An input 
device is caused to follow the path indicated by the 
heavy line 3. The movement is initially straight 
from point 4 to point 5. and then there is an abrupt 
change in direction after which the input device 
initially follows a straight line but then follows an 
arcuate path. The lines 6 indicate the position of 
the input device at successive timing intervals 
(each timing Interval may con^espond to a predeter- 
mined number of position sampling intervals). Thus 
the spacing between adjacent lines 6 along the 
path increases with speed and the change in the 
spacing and the relative inclinations between adja- 
cent lines 6 is a measure of acceleration. In the 
example of Figure 4. the system has been set up 
to "paint" circular shapes the size of which in- 
creases with velocity. This is equivalent to the sort 
of effects one can achieve with a heavily loaded 
dripping brush or fountain pen. (Depending on the 
effect desired, the width/velocity relationship could 
be reversed so that the resultant pattern is thinner 
as the velocity increases. The relationship does not 
have to be directly proportional). The system is 
also set up to simulate splash effects such as 
occurs when a heavily loaded brush is rapidly 
decelerated, e.g. at point 5. The input device was 
decelerated as it approached point 5 from point 4. 
and thus the splash effect 7 is in line with the Initial 
direction of movement of the input device. The 
splash effect is triggered if the calculated accelera- 
tion exceeds a preset threshold determined by the 
media behaviour database 60. 



Figure 5 illustrates the results achieved with a 
system operating as described above with refer- 
ence to Figure 4. it will be seen that the splash 
effect is represented as three "drips" (as shown at 

5 65) positioned in a line orthogonal to the accelera- 
tion direction. 

Figure 6 illustrates how extra effects can be 
achieved. A loaded soft brush which is rapidly 
accelerated exhibits inertial effects as the shape of 

70 the brush changes with acceleration. The effect of 
velocity, acceleration and/or pressure on flexure of 
bristles of the brush can be modelled in the media 
database. In Rgure 6. an input device is assumed 
to have moved along a straight line path from point 

75 8 to point 9. along a curved path from point 9 to 
point 10. and then along a straight line path from 
point 10 to point 11. Shapes 12 made up from 
circular leading sectors and triangular trailing sec- 
tions are "painted" at intervals the spacing be- 

20 tween which is speed dependent. The speed is 
thus roughly constant but the velocity changes. As 
the input device accelerates to move around the 
curve from point 9 to point 10, the trailing end of 
the shape 12 swings out away from the line 9, 10, 

25 the degree to which the shape swings out being a 
function of acceleration perpendicular to the ve- 
locity vector and possibly also the pressure applied 
to the input device. Thus the inertial effects of a 
heavily loaded soft brush can be simulated. The 

30 pattern displayed can be a string of shapes 12. or 
the envelope indicated by lines 13 and 14 swept 
out by the shapes 12 can be filled in. 

Another example of effects which can be 
achieved will now be described with reference to 

35 Figures 8 and 9. In this example, an elliptical brush 
is simulated and the splash effect due to accelera- 
tion modelled. Initially, in a step 70 the media 
behaviour database 60 is loaded with fixed param- 
eters including proportionality constants D1 and k 

40 and thresholds D2 and VC. In operation, the cal- 
culation procedure 51 (Figure 2) is initiated and this 
starts by determining the current coordinates IX lY 
from the tablet over which the pen is moved and 
also determines the pen status (step 71). These 

45 coordinates are sampled at successive, equally 
spaced time intervals determined by the clock 52. 
The previous X coordinate (OLDIX) is obtained 
from a register in a step 72 and then the difference 
between the two X coordinates per unit time is 

60 determined (step 73) to obtain the X velocity 
(ACX). Similarly, the previous Y coordinate (OLDIY) 
is obtained from a register (Step 74), and the 
difference in Y coordinates per unit time is deter- 
mined so as to define the Y velocity (ACY) in a 

55 step 75. The signs of ACX and ACY are examined 
and if these are each positive and both ACX and 
ACY exceed the threshold VC (step 76) then a 
colour index is incremented by one. The colour 
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index is stored in the media database 70 and 
defines a series of different colours which may 
graduate in terms of brightness from a compara- 
tively dark colour to a very bright colour with in- 
creasing colour index value or can be a completely 
artificial set of colours. The effect of this procedure 
is that the faster the pen is moved, the brighter the 
colour becomes. 

In a step 77 it is determined whether the signs 
of the velocities ACX and ACY are both negative 
but yet both ACX and ACY exceed the threshold 
VC in which case the colour index is decreased by 
one, 

A velocity register is updated at each sampling 
interval with the current velocity so that in a step 
78 the previous X velocity is read from the register 
and the difference between the previous X velocity 
and the current X velocity per unit time is deter- 
mined so as to define an acceleration variable 
ACACX (step 79) which is stored. In a similar 
manner the previous Y velocity is read from a 
register (step 80) and the difference between the 
present Y velocity and the previous Y velocity per 
unit time is determined in order to define the Y 
acceleration variable ACACY (step 81 ), 

If the pen state is non-zero (step 82) indicating 
that the splash effect is desired, then in a step 83 it 
is determined whether the X acceleration exceeds 
a threshold 02 and if it does the position of secon- 
dary drips in the X direction is determined. This 
distance is determined to be proportional to the X 
velocity, the proportionality constant being the val- 
ue k previously stored. 

In a similar manner, in a step 84, the Y accel- 
eration is compared with the threshold D2 and the 
displacement of secondary drips in the Y direction 
is determined. 

Following step 84, or where appropriate step 
82, a step 85 is performed in which the size of the 
main spot or primary drip from the brush is deter- 
mined. This size can vary dynamically depending 
on the values of the X and Y velocities and is 
defined as an elipse with major axes in the X and Y 
directions of D1 ACX and D1 ACY respectively 
(Figure 9). 

In a step 86 the coordinates of the centre of 
the primary and secondary drips are determined 
and the real-time dynamic database updated ac- 
cordingly so that pixels corresponding to the pri- 
mary and secondary drips are loaded with a suit- 
able time delay with data defining the drips in 
terms of colour densities with the result that the 
primary and secondary drips are then successively 
displayed (step 87) on the monitor 63. 

Finally, the various registers recording X, Y 
coordinates velocities and accelerations are up- 
dated (step 88) and processing returns (step 89) to 
step 71. 



Figure 9 illustrates a typical form for a primary 
drip 90 and a set of three secondary drips 91. The 
dimensions of the primary drip 90 are defined in 
Rgure 9 and the displacement of the secondary 

5 drips 91 from the Y axis is also defined. In this 
case, it will have been found in step 83 that the 
acceleration variable in the X direction exceeds D2 
but that the acceleration variable in the Y direction 
does not exceed D2. Consequently, the secondary 

70 drips 91 are not displaced in the Y direction. 

It will also be apparent from Figure 9 that 
different effects can be achieved by changing the 
size of the proportionality constants and by using 
different proportionality constants for defining the 

15 size of the primary drip 90 in the X and Y direc- 
tions. 

Referring now to Figure 7, this illustrates ef- 
fects which can be achieved by the interactive 
modelling techniques described with reference to 

20 Figure 3. If we imagine that we start with a horizon- 
tal sheet in the plane defined by X and Y (the 
predefined three-dimensional field model), each op- 
eration of the input device will add to that model 
and layers of material can be built up to define a 

25 three-dimensional surface. Such a surface is in- 
dicated In Figure 7 by lines 15 which lie in respec- 
tive planes that cut the horizontal surface along 
lines 16 parallel to the X axis. The lines 15 ex- 
agerate the thickness of the built-up layers. Each 

30 part of this surface can be assigned a surface 
texture dependent upon the inputs which led to its 
formation, the texture defining "slippiness", that is 
the tendency of the next applied layer to slide over 
the surface. When the input device is placed on a 

35 point 17 which corresponds to the point 18 on the 
built up surface the angle of inclination to the 
horizontal of the surface at point 18 can be cal- 
culated and the resultant image can be adjusted as 
if the material deposited on point 18 slides down 

40 the built up surface in a manner consistent with the 
surface inclination and texture. Thus effects analo- 
gous to the behaviour of flowing materials such as 
paint can be simulated. The viscosity of the flowing 
material would of course be modelled in the media 

4$ database. This can be important where relatively 
large amounts of "paint" are to be introduced as a 
result of drop and splash effects related to the 
velocity and acceleration of the input device. 

Further effects to those described can be 

50 achieved, for example textures dependent upon 
velocity, colours which are dependent on velocity, 
and the like. The effects do not necessarily have to 
emulate effects found with traditional media. 

56 

ClainDs 

1 . A graphics system comprising an input de- 
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vice, means for detecting changes in the position 
of the input device relative to a surface, and means 
for displaying patterns which follow the nriovement 
of the input device, wherein means are provided for 
determining at least one differential derivative with 5 
respect to time of the Input device position, and 
means are provided for controlling characteristics 
of the displayed patterns in dependence upon the 
or each determined derivative. 

2. A system according to claim 1, wherein the io 
components defining the Input device position 
comprise first and second components defining the 
displacement of the input device and a third com- 
ponent defining the pressure with which the device 

is applied to the surface or the displacement of the 75 
device away from the surface. 

3. A system according to claim 1 or claim 2, 
wherein the means for determining at least one 
differential derivative with respect to time is adapt- 
ed to calculate the derivative of one or more of the 20 
components defining the input device position. 

4. A system according to any of the preceding 
claims, wherein the determined derivatives control 
one or more of the shape, width, density, texture 

and colour of the displayed patterns. 26 

5. A system according to any of the preceding 
claims, wherein the acceleration of the Input device 
relative to the surface is determined, the means for 
controlling characteristics of the display pattern de- 
termining whether the acceleration exceeds a 3o 
threshold and, if it does, causing a splash effect to 

be displayed. 

6. A graphics system comprising an input de- 
vice, means for detecting changes in the position 

of the input device relative to a surface, means for 35 
displaying patterns which follow the movement of 
the input device, and storage means for storing 
data defining the form of a simulated surface on , 
which the patterns are being drawn, wherein the 
means for displaying patterns is responsive to 40 
movement of the input device and the stored data 
to simulate the effect of underlying surface on the 
pattern being drawn. 

7. A system according to claim 6, wherein the 
stored data simulates the thickness of a simulated 45 
colour medium laid down previously. 

8. A system according to any of claims 1 to 6 
and claim 7 or claim 8. 
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